Motion imaging systems have been in use for over one hundred years. These systems rely upon the capture and presentation of a sequential series of still images. If the sequence is presented at a sufficiently rapid rate, the human eye and brain does not perceive the images as separate, still images but rather as a smooth sequence of moving images similar to what is perceived in the real world. The minimum rate at which the images are captured or presented varies somewhat in different imaging systems. For example, traditional motion picture film is typically acquired and presented at 24 frames per second. Standard NTSC television presents 30 frames per second while PAL uses 25. Image sequences are sometimes acquired at one rate and presented at another for artistic effect, for example to create slow motion effects.
Higher presentation frame rates can provide more pleasing presentation of changes in scene content (such as, pans, object motion, or scene changes) by increasing the observable detail in the frames, reducing blur, and smoothing the motion. Thus, the frame rate necessary to maintain the illusion of smooth motion will depend on the nature and content of the scene. The situation is complicated by the viewer's behavior. For example, the amount and location of detail that is perceived in a motion sequence depends, at least in part, on whether the viewer's eyes track a moving object in the sequence. Hence, the necessary frame rate for optimal viewing of a motion image sequence depends on the nature of the imagery and the behavior of the viewer as well as the viewing environment.
Today, motion imaging can be realized with either traditional motion picture films or with electronic imaging devices, either analog or digital. Digital motion image sequences are very profligate with computing resources, requiring huge amounts of storage, high-resolution capture and presentation devices, massive computational needs, and a large bandwidth for communication. Since the amount of image data is dependent to a very great extent upon the frame rate, any digital imaging system requires a balance between the amount of image data and the perceived motion image quality.
The modification of a motion sequence frame rate for both capture and display is known in the prior art. U.S. Pat. No. 3,723,644 issued Mar. 27, 1973 to Haskell et al. describes a system for analyzing the motion content of an imaging sequence and adjusting the frame capture rate to complement the motion detected. Frames are counted and the count used to specify the frame repetition rate at playback thus reducing transmission bandwidth requirements. WO 99/67590 published Dec. 29, 1999 describes varying the presentation frame rate of an image sequence to prevent piracy. Tradeoffs between frame rate and image detail are also known. U.S. Pat. No. 5,389,965 issued Feb. 14, 1995 to Kuzma describes an image sequence transmitted over a fixed bandwidth channel wherein the frame rate is increased when image detail is decreased.
Image sequences acquired at one fixed frame rate can be converted and displayed at another frame rate. For example, film-based image sequences acquired at 24 frames per second can be displayed at 30 frames per second for television broadcast in real-time. These conversions are achieved either by replicating (repeating) frames or by temporally interpolating new frames between existing ones.
Today, image sequences are captured at a fixed frame rate although post-processing can reduce the frame rate for compression or communication purposes. The cost of increasing the capture frame rate is prohibitive, particularly for motion picture film, since the amount of film used is increased according to the increase in frame rate. Moreover, even if the image sequences are acquired or stored digitally at a higher rate, the concomitant increase in computing resources, communication bandwidth, etc. is problematic.
There is a need, therefore, for a motion imaging system that optimizes the motion quality and minimizes the computing resources needed to store, manipulate, and communicate a digital motion image sequence.